{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Stable Diffusion XL Turbo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SDXL Turbo is an adversarial time-distilled [Stable Diffusion XL](https://huggingface.co/papers/2307.01952) (SDXL) model capable\n",
    "of running inference in as little as 1 step.\n",
    "\n",
    "This guide will show you how to use SDXL-Turbo for text-to-image and image-to-image.\n",
    "\n",
    "Before you begin, make sure you have the following libraries installed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# uncomment to install the necessary libraries in Colab\n",
    "#!pip install -q diffusers transformers accelerate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load model checkpoints"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Model weights may be stored in separate subfolders on the Hub or locally, in which case, you should use the [from_pretrained()](https://huggingface.co/docs/diffusers/main/en/api/pipelines/overview#diffusers.DiffusionPipeline.from_pretrained) method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from diffusers import AutoPipelineForText2Image\n",
    "import torch\n",
    "\n",
    "pipeline = AutoPipelineForText2Image.from_pretrained(\"stabilityai/sdxl-turbo\", torch_dtype=torch.float16, variant=\"fp16\")\n",
    "pipeline = pipeline.to(\"cuda\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also use the [from_single_file()](https://huggingface.co/docs/diffusers/main/en/api/loaders/single_file#diffusers.loaders.FromSingleFileMixin.from_single_file) method to load a model checkpoint stored in a single file format (`.ckpt` or `.safetensors`) from the Hub or locally. For this loading method, you need to set `timestep_spacing=\"trailing\"` (feel free to experiment with the other scheduler config values to get better results):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from diffusers import StableDiffusionXLPipeline, EulerAncestralDiscreteScheduler\n",
    "import torch\n",
    "\n",
    "pipeline = StableDiffusionXLPipeline.from_single_file(\n",
    "    \"https://huggingface.co/stabilityai/sdxl-turbo/blob/main/sd_xl_turbo_1.0_fp16.safetensors\",\n",
    "    torch_dtype=torch.float16, variant=\"fp16\")\n",
    "pipeline = pipeline.to(\"cuda\")\n",
    "pipeline.scheduler = EulerAncestralDiscreteScheduler.from_config(pipeline.scheduler.config, timestep_spacing=\"trailing\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Text-to-image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For text-to-image, pass a text prompt. By default, SDXL Turbo generates a 512x512 image, and that resolution gives the best results. You can try setting the `height` and `width` parameters to 768x768 or 1024x1024, but you should expect quality degradations when doing so.\n",
    "\n",
    "Make sure to set `guidance_scale` to 0.0 to disable, as the model was trained without it. A single inference step is enough to generate high quality images.\n",
    "Increasing the number of steps to 2, 3 or 4 should improve image quality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from diffusers import AutoPipelineForText2Image\n",
    "import torch\n",
    "\n",
    "pipeline_text2image = AutoPipelineForText2Image.from_pretrained(\"stabilityai/sdxl-turbo\", torch_dtype=torch.float16, variant=\"fp16\")\n",
    "pipeline_text2image = pipeline_text2image.to(\"cuda\")\n",
    "\n",
    "prompt = \"A cinematic shot of a baby racoon wearing an intricate italian priest robe.\"\n",
    "\n",
    "image = pipeline_text2image(prompt=prompt, guidance_scale=0.0, num_inference_steps=1).images[0]\n",
    "image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"flex justify-center\">\n",
    "    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/sdxl-turbo-text2img.png\" alt=\"generated image of a racoon in a robe\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Image-to-image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For image-to-image generation, make sure that `num_inference_steps * strength` is larger or equal to 1.\n",
    "The image-to-image pipeline will run for `int(num_inference_steps * strength)` steps, e.g. `0.5 * 2.0 = 1` step in\n",
    "our example below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from diffusers import AutoPipelineForImage2Image\n",
    "from diffusers.utils import load_image, make_image_grid\n",
    "\n",
    "# use from_pipe to avoid consuming additional memory when loading a checkpoint\n",
    "pipeline_image2image = AutoPipelineForImage2Image.from_pipe(pipeline_text2image).to(\"cuda\")\n",
    "\n",
    "init_image = load_image(\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png\")\n",
    "init_image = init_image.resize((512, 512))\n",
    "\n",
    "prompt = \"cat wizard, gandalf, lord of the rings, detailed, fantasy, cute, adorable, Pixar, Disney, 8k\"\n",
    "\n",
    "image = pipeline_image2image(prompt, image=init_image, strength=0.5, guidance_scale=0.0, num_inference_steps=2).images[0]\n",
    "make_image_grid([init_image, image], rows=1, cols=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"flex justify-center\">\n",
    "    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/sdxl-turbo-img2img.png\" alt=\"Image-to-image generation sample using SDXL Turbo\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Speed-up SDXL Turbo even more"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Compile the UNet if you are using PyTorch version 2.0 or higher. The first inference run will be very slow, but subsequent ones will be much faster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe.unet = torch.compile(pipe.unet, mode=\"reduce-overhead\", fullgraph=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- When using the default VAE, keep it in `float32` to avoid costly `dtype` conversions before and after each generation. You only need to do this one before your first generation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe.upcast_vae()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As an alternative, you can also use a [16-bit VAE](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix) created by community member [`@madebyollin`](https://huggingface.co/madebyollin) that does not need to be upcasted to `float32`."
   ]
  }
 ],
 "metadata": {},
 "nbformat": 4,
 "nbformat_minor": 4
}
